Imaging apparatus and imaging method for maintaining continuity between frames of moving image data

ABSTRACT

An imaging apparatus includes an imaging unit, a first image processor, and a second image processor. The imaging unit sequentially acquires image frames whose photography conditions have been changed. The first image processor generates moving image data by performing image processing to maintain continuity between the image frames obtained by changing the photography conditions. The second image processor composes the image frames whose photography conditions have been changed, to generate condition-changed still image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2016-176758, filed Sep. 9,2016, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of Invention

The present invention relates to an imaging apparatus and an imagingmethod.

2. Description of Related Art

Recent imaging apparatuses are each connectable to a television by wiredcommunication or wireless communication. Thus, a user can watch imagesacquired by the imaging apparatus in a large screen of the television.Imaging apparatuses can be apparatuses which can provide attractivecontents to televisions.

For example, an imaging apparatus suggested in Jpn. Pat. Appln. KOKAIPublication No. 2015-56807 composes images having different exposureconditions obtained during the recording of an HDR moving image whenthere is given an instruction of a still image photography during therecording of the HDR moving image, thereby recording an HDR still image.

BRIEF SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided animaging apparatus comprising: an imaging unit which sequentiallyacquires image frames whose photography conditions have been changed; afirst image processor which generates moving image data by performingimage processing to maintain continuity between the image framesobtained by changing the photography conditions; and a second imageprocessor which composes the image frames whose photography conditionshave been changed, to generate condition-changed still image data.

According to a second aspect of the invention, there is provided animaging apparatus comprising: an imaging unit which sequentiallyacquires image data groups comprising pieces of image data different inphotography conditions; a first image processor which generates movingimage data comprising moving image frames formed by each of the piecesof the image data; and a second image processor which composes pieces ofimage data belonging to each of the image data groups or pieces of imagedata that do not belong to the image data groups, to generatecondition-changed still image data.

According to a third aspect of the invention, there is provided animaging method comprising: sequentially acquiring, by an imaging unit,image frames whose photography conditions have been changed; generatingmoving image data by performing image processing to maintain continuitybetween the image frames obtained by changing the photographyconditions; and composing the image frames whose photography conditionshave been changed, to generate condition-changed still image data.

According to a fourth aspect of the invention, there is provided animaging method comprising: sequentially acquiring, by an imaging unit,image data groups comprising pieces of image data different inphotography conditions; generating moving image data comprising movingimage frames formed by each of the pieces of the image data; andcomposing image data belonging to each of the image data groups or imagedata that do not belong to the image data groups, to generatecondition-changed still image data.

Advantages of the invention will be set forth in the description whichfollows, and in part will be obvious from the description, or may belearned by practice of the invention. The advantages of the inventionmay be realized and obtained by means of the instrumentalities andcombinations particularly pointed out hereinafter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the general description given above and the detaileddescription of the embodiments given below, serve to explain theprinciples of the invention.

FIG. 1 is a block diagram showing the configuration of one example of animaging apparatus according to one embodiment of the present invention;

FIG. 2A is a flowchart showing the main processing of the imagingapparatus according to one embodiment of the present invention;

FIG. 2B is a flowchart showing the main processing of the imagingapparatus according to one embodiment of the present invention;

FIG. 3A is a diagram showing a display example of operation mode icons;

FIG. 3B is a diagram showing a display example of through-image displayand photography mode icons;

FIG. 3C is a diagram showing a display example of a setting screen;

FIG. 4 is a flowchart showing processing of an effect check;

FIG. 5A is a diagram showing an example of display of a list of imagedata;

FIG. 5B is a diagram showing an example of comparison display;

FIG. 5C is a diagram showing an example of enlarged display;

FIG. 6 is a diagram showing an example of composition processing;

FIG. 7A is a timing chart showing processing of condition-changed movingimage photography set to generate an HDR still image;

FIG. 7B is a timing chart showing processing of condition-changed movingimage photography set to generate a super-resolution still image;

FIG. 8A is a diagram showing the structure of a moving image file;

FIG. 8B is a diagram showing the structure of a condition-changed stillimage file;

FIG. 9 is a diagram showing the structure of a moving image file in thecase where moving image data and still image data are recorded in onefile;

FIG. 10A is a flowchart showing other processing;

FIG. 10B is a flowchart showing the other processing; and

FIG. 11 is a diagram showing an example of playback of an image in anexternal display device.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an embodiment of the present invention will be describedwith reference to the drawings. FIG. 1 is a block diagram showing theconfiguration of one example of an imaging apparatus according to oneembodiment of the present invention. An imaging apparatus 100 shown inFIG. 1 includes various devices having imaging functions, such as adigital camera, a smartphone, and a mobile telephone having a camerafunction. The imaging apparatus 100 shown in FIG. 1 has an imaging unit102, a storage unit 104, an image processor 106, a playback unit 108, adisplay 110, a recording unit 112, a communication unit 114, anoperation unit 116, a controller 118, and a microphone 120. Here, eachblock of the imaging apparatus 100 is formed by, for example, acombination of a hardware and a software. Each block of the imagingapparatus 100 does not need to be formed by a single hardware orsoftware, and may be formed by hardwares and softwares.

The imaging unit 102 includes an imaging optical system 102 a, animaging element 102 b, and a drive unit 102 c. The imaging opticalsystem 102 a includes an aperture, lenses, and others, and allows alight flux from an unshown subject to enter the imaging element 102 b.The imaging optical system 102 a also includes a focus lens to adjust afocus state. The imaging element 102 b includes, for example, a CMOSimage sensor or a CCD image sensor, and images a target and acquiresimage data (RAW data) regarding the target. The imaging element 102 bmay include phase difference detection pixels so that the distance tothe photography target can be detected. The imaging element 102 baccording to the present embodiment may be configured to be movable in aplane orthogonal to an optical axis of the imaging optical system 102 a.The drive unit 102 c drives the focus lens of the imaging optical system102 a in its optical axis direction, or drives the imaging element 102b, under the control of the controller 118.

The storage unit 104 is, for example, a DRAM, and transitorily storesimage data acquired by the imaging unit 102.

The image processor 106 subjects, to image processing, the image dataacquired by the imaging unit 102 and then stored in the storage unit104. The image processor 106 includes a first image processor 106 a, asecond image processor 106 b, and a record processor 106 c.

The first image processor 106 a subjects the image data stored in thestorage unit 104 to image processing regarding the generation of movingimage data. Although described later, image data different inphotography conditions are generated during moving image photography inthe present embodiment. The first image processor 106 a performsprocessing to maintain continuity between moving image frames beforegenerating moving image data from the image data different inphotography conditions. This processing to maintain continuity includesgain adjustment processing, blurring processing, contrast enhancementprocessing, and others.

The second image processor 106 b subjects the image data stored in thestorage unit 104 to image processing regarding the generation of stillimage data. Here, the still image data according to the presentembodiment includes normal still image data and condition-changed stillimage data. The normal still image data is still image data generatedfrom one piece of image data acquired by the imaging unit 102. Thecondition-changed still image data is still image data formed by thecomposition of the pieces of image data different in photographyconditions acquired by the imaging unit 102. The condition-changed stillimage data includes, for example, HDR still image data, super-resolutionstill image data, depth-composed still image data, and panning effectstill image data.

The record processor 106 c performs processing to generate an image filefor recording from the image data generated by the first image processor106 a or the second image processor 106 b. For example, the recordprocessor 106 c compresses the moving image data in a predeterminedmoving image compression scheme such as an H.264 scheme, and providespredetermined tag information to the compressed moving image data, togenerate a moving image file. Alternatively, for example, the recordprocessor 106 c compresses the still image data in a predetermined stillimage compression scheme such as a JPEG scheme, and providespredetermined tag information to the compressed still image data, togenerate a still image file. Here, the record processor 106 c performsprocessing to associate the still image file with the moving image filewhen the still image file is generated during moving image photography.For example, the record processor 106 c records, as tag information,information (e.g. the file name of an associated file) to associate thestill image file with the moving image file.

The playback unit 108 performs processing to play back the image datagenerated by the image processor 106. The playback unit 108 has a firstplayback processor 108 a, and a second playback processor 108 b.

The first playback processor 108 a performs processing to play back themoving image data in the display 110 or the television 200. For example,the first playback processor 108 a inputs the moving image datagenerated in the first image processor 106 a to the display 110 so thatimages will be displayed in the display 110. The first playbackprocessor 108 a sends the moving image data generated in the first imageprocessor 106 a to the television 200 via the communication unit 114 sothat images will be displayed in the television 200. When the image datais compressed, the first playback processor 108 a also decompresses thecompressed image data. This image processing affects smoothness ofmoving images and is therefore preferably high-speed processing.Instantaneousness is also important in through-images or the like. Inthe case of moving images, smoothness of a change is required, so thateven control with delayed response or the like is also suitablyperformed in consideration of continuity of preceding and subsequentframes.

The second playback processor 108 b performs processing to play back thestill image data in the display 110 or the television 200. For example,the second playback processor 108 b inputs the still image datagenerated in the second image processor 106 b to the display 110 so thatimages will be displayed in the display 110. The second playbackprocessor 108 b sends the still image data generated in the second imageprocessor 106 b to the television 200 via the communication unit 114 sothat images will be displayed in the television 200. When the image datais compressed, the second playback processor 108 b also decompresses thecompressed image data. In the case of a still image, evaluation is madewith the relevant image alone independently of preceding and subsequentframes, so that such an expression as in an image in the process of ascene response of moving images is not preferable. That is, even if animage is an underexposure image in the stream in moving images, a stillimage obtained by this timing needs to have correct exposure. However,because this image processing is only applied to limited frames,temporal restriction and instantaneousness are relatively not required,and the appropriateness, decency, and the like of the expression of animage to be obtained are more important than time. Moreover, a stillimage is also required to adapt to printing and the like, in which caseprocessing to adapt to the input specifications and standard of aprinter may be additionally involved. To this end, image data forprinting may be separately provided.

The display 110 is, for example, a liquid crystal display or an organicEL display, and displays various images such as images based on theimage data input from the playback unit 108.

The recording unit 112 comprises, for example, a flash ROM. Image filesgenerated in the record processor 106 c of the image processor 106 arerecorded in the recording unit 112. Various programs used to control theimaging apparatus 100 may also be recorded in the recording unit 112.

The communication unit 114 performs processing to mediate acommunication between the imaging apparatus 100 and the television 200which is an external display device. The communication unit 114 mediatesthe communication between the imaging apparatus 100 and the television200, for example, by HDMI. The communication unit 114 may mediate thecommunication between the imaging apparatus 100 and the television 200by wireless communication such as WiFi. The communication unit 114 mayalso be configured to be able to communicate with devices other than theexternal display device.

The operation unit 116 is an operational component for a user to operatethe imaging apparatus 100. The operation unit 116 includes, for example,a release button, a moving image button, a setting button, a selectionkey, an electric power button, a touch panel, and others. The releasebutton is an operational component to issue an instruction for stillimage photography. The moving image button is an operational componentto instruct to start or end moving image photography. The setting buttonis an operational component to display a setting screen for the imagingapparatus 100. The selection key is an operational component to selectand decide an item on, for example, the setting screen. The electricpower button is an operational component to turn on or off the electricpower of the imaging apparatus 100. The touch panel is providedintegrally with a display screen of the display 110, and detects auser's touch operation on the display screen. The touch panel may becapable of operations equivalent to those of the release button, themoving image button, the setting button, the selection key, and theelectric power button described above. Moreover, the operation unit 116may also have other operational components in addition to theoperational components described above.

The controller 118 is a control circuit such as a CPU and an ASIC, andhas overall control of the operation of the imaging apparatus 100.Functions equivalent to those of the controller 118 may be enabled by asoftware, or may be enabled by a combination of a hardware and asoftware. Some of the functions of the controller 118 may be providedseparately from the controller 118.

The microphone 120 acquires input sound after converting the sound intoan electric signal.

The television 200 which is an external display device displays, forexample, television videos. The television 200 also displays images onthe basis of the image data transferred from the communication unit 114.

Now, the operation of the imaging apparatus according to the presentembodiment is described. FIG. 2A and FIG. 2B are flowcharts showing themain processing of the imaging apparatus according to the presentembodiment. The processing in FIG. 2A and FIG. 2B is performed mainly bythe controller 118.

The processing in FIG. 2A and FIG. 2B is started when the electric powersupply is turned on. In step S1, the controller 118 instructs theplayback unit 108 to display operation mode icons on the display 110 forthe user to select an operation mode of the imaging apparatus 100. Inresponse to this instruction, the playback unit 108 displays, forexample, a photography icon 301, a playback icon 302, and acommunication icon 303 shown in FIG. 3A. The photography icon 301 is anicon for the user to instruct to switch the operation mode of theimaging apparatus 100 to a photography mode. The playback icon 302 is anicon for the user to instruct to switch the operation mode of theimaging apparatus 100 to a playback mode. The communication icon 303 isan icon for the user to instruct to switch the operation mode of theimaging apparatus 100 to a communication mode. Icons corresponding tooperation modes other than the photography icon 301, the playback icon302, and the communication icon 303 may be displayed. Otherwise, theicon display shown in FIG. 3A may not be performed. In this case, forexample, when the electric power supply is turned on, the imagingapparatus 100 may enter an operation mode immediately before theelectric power supply off, or enter a predetermined operation mode amongthe photography mode, the playback mode, and the communication mode.

In step S2, the controller 118 judges whether or not the currentoperation mode is the photography mode. That is, the controller 118judges whether or not the current operation mode is the photographymode, by judging which icon is selected by the user. When it is judgedin step S2 that the operation mode of the imaging apparatus 100 is thephotography mode, the processing proceeds to step S3. When it is judgedin step S2 that the operation mode of the imaging apparatus 100 is notthe photography mode, the processing proceeds to step S26.

In step S3, the controller 118 causes the imaging unit 102 to performimaging to acquire image data for through-image display. In step S4, thecontroller 118 performs the through-image display. As the through-imagedisplay, the controller 118 causes the image processor 106 to performsignal processing. Accordingly, the image processor 106 subjects theimage data acquired in the imaging unit 102 and then stored in thestorage unit 104 to the image processing (the white balance correction,the gradation conversion, the color correction, and others) necessaryfor the through-image display to generate through-image data. After thegeneration of the through-image data, the playback unit 108 sequentiallydisplays through-images 304 based on the generated through-image data onthe display 110 as shown in FIG. 3B. The playback unit 108 alsodisplays, for example, a setting icon 305, a moving image photographyicon 306, a still image photography icon 307, and a return icon 308shown in FIG. 3B in, for example, a display region different from adisplay region of the through-images 304. The setting icon 305 is anicon for the user to issue a setting instruction. The moving imagephotography icon 306 is an icon for the user to instruct to start or endmoving image photography. The still image photography icon 307 is anicon for the user to issue an instruction for still image photography.The return icon 308 is an icon for the user to instruct to end thephotography mode. These icons for the photography modes may be displayedover the through-images 304. Icons other than the setting icon 305, themoving image photography icon 306, the still image photography icon 307,and the return icon 308 may be displayed as icons for the photographymodes.

In step S5, the controller 118 judges whether or not the user hasinstructed to set the imaging apparatus 100. For example, when the userhas selected the setting icon 305 or operated the setting button, it isjudged that a setting instruction has been issued. When it is judged instep S5 that the setting instruction has been issued, the processingproceeds to step S6. When it is judged in step S5 that the settinginstruction has not been issued, the processing proceeds to step S7.

In step S6, the controller 118 instructs the playback unit 108 todisplay the setting screen. In response to this instruction, theplayback unit 108 displays, for example, the setting screen shown inFIG. 3C on the display 110. After the display of the setting screen, thecontroller 118 changes various settings in accordance with the user'soperation on the setting screen. After the end of the setting on thesetting screen, the processing proceeds to step S7. In the example ofFIG. 3C, a photography setting icon 309, a condition-changed recordingicon 310, and a return icon 311 are displayed on the setting screen. Thephotography setting icon 309 is an icon for the user to perform varioussettings at the time of photography. By selecting the photographysetting icon 309, the user can set exposure conditions at the time ofphotography (set a shutter speed and an aperture), set recorded imagequality at the time of photography, and others. The condition-changedrecording icon 310 is an icon for the user to perform various settingsregarding a condition-changed recording mode. By selecting thecondition-changed recording icon 310, the user can set the on/off of thecondition-changed recording mode, set the recording timing for thecondition-changed still image during moving image photography, and makean effect check to perform setting regarding the composition processingof the condition-changed still image. The return icon 311 is an icon forthe user to instruct to end the display of the setting screen.

The effect check is described below. FIG. 4 is a flowchart showingprocessing of the effect check. In the effect check, the user canperform various settings regarding the condition-changed still imagedata generated during the moving image photography while actuallychecking the condition-changed still image.

In step S101, the controller 118 acquires image data for the effectcheck. In the effect check in the photography mode, the controller 118controls the imaging unit 102 so that an image data group comprisingpieces of image data different in exposure conditions will be acquired.For example, when an effect check of an HDR still image as acondition-changed still image is made, the controller 118 controls theimaging unit 102 so that an image data group comprising pieces of imagedata different in exposure amount will be acquired. For example, when aneffect check of a super-resolution still image as a condition-changedstill image is made, the controller 118 controls the imaging unit 102 sothat an image data group comprising pieces of image data different inthe shift direction of the imaging element 102 b will be acquired.Further, when an effect check of a depth-composed still image as acondition-changed still image is made, the controller 118 controls theimaging unit 102 so that an image data group comprising pieces of imagedata different in focus lens position will be acquired. Moreover, whenan effect check of a panning effect still image as a condition-changedstill image is made, the controller 118 controls the imaging unit 102 sothat an image data group comprising pieces of image data different inimaging timing will be acquired. In the processing in step S101immediately after the execution of the processing of the effect check,the number of times of imaging and the change amount of conditions arefixed values.

In step S102, the controller 118 controls the playback unit 108 todisplay a list of the acquired image data on the display 110. Inresponse to this instruction, the playback unit 108 displays reducedimages 401, 402, and 403 of the image data on the display 110, forexample, as shown in FIG. 5A. Then the processing proceeds to step S103.The example in FIG. 5A is a display example in the case where an HDReffect check is made. Here, the reduced image 401 is a reduced image ofthe image data obtained by the imaging under the condition of the normalexposure, the reduced image 402 is a reduced image of the image dataobtained by the imaging under the condition of the overexposure, and thereduced image 403 is a reduced image of the image data obtained by theimaging under the condition of the underexposure. Further, the playbackunit 108 displays, for example, an adjustment icon 404, an effect checkicon 405, and a return icon 406. The adjustment icon 404 is an icon forthe user to adjust composition processing. The effect check icon 405 isan icon for the user to check the result of the composition processing.The return icon 406 is an icon for the user to instruct to end theeffect check processing.

In step S103, the controller 118 judges whether or not to make anadjustment. For example, it is judged that an adjustment will be madewhen the user has selected the adjustment icon 404. When it is judged instep S103 that an adjustment will be made, the processing proceeds tostep S104. When it is judged in step S103 that an adjustment will not bemade, the processing proceeds to step S105.

In step S104, the controller 118 makes an adjustment regarding thecomposition processing in the effect check in accordance with the user'soperation. In this adjustment, the user can adjust the number of imagedata for use in the composition processing, the exposure difference ofthe image data, and others, in the effect check of the still image.After the end of the adjustment, the processing returns to step S101.Here, when image data need to be again acquired due to the adjustment,for example, when the number of image data for use in the compositionprocessing is more than an initial value and when there is a change inthe exposure difference of the image data, image data are again acquiredin step S101, and the display in step S102 is updated.

In step S105, the controller 118 judges whether or not the image datafor use in the composition processing has been selected by the user. Forexample, when the display shown in FIG. 5A is performed, the user canselect the image data for use in the composition processing by touchingthe reduced images 401, 402, and 403. In step S105, when the number ofimage data necessary for the composition processing are selected by theuser, it is judged that the image data for use in the compositionprocessing have been selected by the user. When it is judged in stepS105 that the image data for use in the composition processing have beenselected by the user, the processing proceeds to step S106. When it isjudged in step S105 that the image data for use in the compositionprocessing have not been selected by the user, the processing proceedsto step S107.

In step S106, the controller 118 judges whether or not to make an effectcheck. For example, when the effect check icon 405 is selected by theuser, it is judged that the effect check will be made. When it is judgedin step S106 that the effect check will not be made, the processingproceeds to step S107. When it is judged in step S106 that the effectcheck will be made, the processing proceeds to step S108.

In step S107, the controller 118 judges whether or not a returninstruction is issued by the user. For example, when the return icon 406is selected by the user, it is judged that the return instruction isissued. When it is judged in step S107 that the return instruction isnot issued by the user, the processing returns to step S101. When it isjudged in step S107 that the return instruction is issued by the user,the processing in FIG. 4 ends.

In step S108, the controller 118 causes the second image processor 106 bof the image processor 106 to perform the second composition processing.In response to this instruction, the second image processor 106 bcomposes the image data selected by the user to generatecondition-changed still image data.

In step S109, the controller 118 controls the playback unit 108 toperform comparison display. In response to this instruction, theplayback unit 108 displays, on the display 110, a list of reduced images407 and 408 of the image data used in the composition processing and areduced image 409 of the image data obtained as a result of thecomposition processing, for example, as shown in FIG. 5B. Further, theplayback unit 108 displays, for example, a reflection icon 410, and areturn icon 411. Then the processing proceeds to step S110. Here, thereflection icon 410 is an icon for the user to instruct to reflect theresult of the composition processing at the time of the followingphotography. The return icon 411 is an icon for the user to instruct toreturn to the screen in FIG. 5A.

For example, in the HDR composition processing, image data having a widedynamic range can be acquired by the composition of image data differentin exposure conditions. Depending on how the image data are selected andhow the composition ratio is set here, the finish of composed image datavaries.

The composition of two of image data 501 of normal exposure, image data502 of overexposure, and image data 503 of underexposure shown in FIG. 6is shown by way of example. For example, in the HDR compositionprocessing using the image data 502 of overexposure and the image data503 of underexposure, the second image processor 106 b performscomposition processing so that a higher composition ratio of the imagedata 503 of underexposure is set for a saturated sky part of the imagedata 502 of overexposure and so that a higher composition ratio of theimage data 502 of overexposure is set for a tree part of blocked upshadows in the image data 503 of underexposure. As a result of suchcomposition processing, HDR still image data 504 having a dynamic rangeextended to a high-luminance side and a low-luminance side is generated.In the HDR still image data 504, colors of both the sky part which is ahigh-luminance part and the tree part which is a low-luminance part canbe reproduced at the same time. However, color reproduction of amountain part which is a medium gradation luminance part is differentfrom color reproduction of a mountain part in the image data of normalexposure.

In the HDR composition processing using the image data 501 of normalexposure and the image data 503 of underexposure, the second imageprocessor 106 b performs composition processing so that a highercomposition ratio of the image data 501 of normal exposure is set forthe mountain part and tree part of blocked up shadows in the image data503 of underexposure and so that a higher composition ratio of the imagedata 503 of underexposure is set for the sky part of blown outhighlights in the image data 501 of normal exposure. As a result of suchcomposition processing, HDR still image data 505 having a dynamic rangeextended to the high-luminance side is generated. In the HDR still imagedata 505, colors of the sky part which is a high-luminance part andrelatively bright parts such as the mountain part which is a mediumluminance part can be reproduced. However, the tree part which is thelow-luminance part has blocked up shadows.

In the HDR composition processing using the image data 501 of normalexposure and the image data 502 of overexposure, the second imageprocessor 106 b performs composition processing so that a highercomposition ratio of the image data 502 of overexposure is set for thetree part of blocked up shadows in the image data 501 of normal exposureand so that a higher composition ratio of the image data 501 of normalexposure is set for the mountain part of blown out highlights in theimage data 502 of overexposure. As a result of such compositionprocessing, HDR still image data 506 having a dynamic range extended tothe low-luminance side is generated. In the HDR still image data 506,colors of the tree part which is the low-luminance part and relativelydark parts such as the mountain part which is the medium luminance partcan be reproduced. However, the sky part which is the high-luminancepart has blown out highlights.

As above, in the composition processing, the finish of the composedimage data varies depending on how the image data are selected.Moreover, the finish of the composed image data also varies by thechange of the composition ratio. When the comparison display in FIG. 5Bis performed in the effect check, the user can set the compositionprocessing for the condition-changed still image data while comparingthe images before and after the composition processing.

Here, the explanation returns to FIG. 4. In step S110, the controller118 judges whether or not one of the reduced images is selected by theuser. When it is judged in step S110 that one of the reduced images isselected by the user, the processing proceeds to step S111. When it isjudged in step S110 that one of the reduced images is not selected bythe user, the processing proceeds to step S115.

In step S111, the controller 118 instructs the playback unit 108 toperform enlarged display of the selected reduced image. In response tothis instruction, the playback unit 108 displays, on the display 110, anenlarged image of the image data corresponding to the selected reducedimage, as shown in FIG. 5C. For example, FIG. 5C shows a display exampleof an enlarged image 409 a of the reduced image 409. Further, theplayback unit 108 displays, for example, a correction icon 412, and areturn icon 413. Then the processing proceeds to step S112. Here, thecorrection icon 412 is an icon for the user to instruct to correct theresult of the composition processing. The return icon 413 is an icon forthe user to instruct to return to the screen in FIG. 5B.

In step S112, the controller 118 judges whether or not to make acorrection. For example, it is judged that a correction will be madewhen the user has selected the correction icon 412. When it is judged instep S112 that a correction will not be made, the processing proceeds tostep S113. When it is judged in step S112 that a correction will bemade, the processing proceeds to step S114.

In step S113, the controller 118 judges whether or not a returninstruction is issued by the user. For example, when the return icon 413is selected by the user, it is judged that the return instruction isissued. When it is judged in step S113 that the return instruction isnot issued by the user, the processing returns to step S111. In thiscase, the enlarged display is continued. When it is judged in step S113that the return instruction is issued by the user, the processingreturns to step S108. In this case, the display returns to the displayin FIG. 5B. When the correction described in step S114 is made beforethe return icon 413 is selected, the contents of the correction are alsoreflected in the display in FIG. 5B.

In step S114, the controller 118 causes the second image processor 106 bto make a correction that conforms to the operation by the user. Thecorrection here includes contrast enhancement, chroma enhancement, andthe provision of special effects by the combination of the above for theselected image data, the change of the composition ratio of the imagedata for use in the composition processing, and others. In the presentembodiment, both the correction of each of the image data beforecomposition and the correction of the image data after composition canbe made. After the end of the correction, the processing returns to stepS111. When the correction is made, composition processing is againperformed in step S111, and the display is then updated.

In step S115, the controller 118 judges whether or not to reflect thecontents of the correction. For example, when the reflection icon 410 isselected by the user, it is judged that the contents of the correctionwill be reflected. When it is judged in step S115 that the contents ofthe correction will not be reflected, the processing proceeds to stepS116. When it is judged in step S115 that the contents of the correctionwill be reflected, the processing proceeds to step S117.

In step S116, the controller 118 judges whether or not a returninstruction is issued by the user. For example, when the return icon 411is selected by the user, it is judged that the return instruction isissued. When it is judged in step S116 that the return instruction isnot issued by the user, the processing returns to step S108. When it isjudged in step S116 that the return instruction is issued by the user,the processing returns to step S101. In this case, image data are againacquired as needed.

In step S117, the controller 118 stores the contents of the correctionin, for example, the storage unit 104. Then the processing in FIG. 4ends. The contents of the correction stored here will be reflected atthe time of the recording of the condition-changed still image.

Here, the explanation returns to FIG. 2A and FIG. 2B. In step S7, thecontroller 118 judges whether or not the condition-changed recordingmode is on. When it is judged in step S7 that the condition-changedrecording mode is on, the processing proceeds to step S8. When it isjudged in step S7 that the condition-changed recording mode is not on,the processing proceeds to step S16.

In step S8, the controller 118 judges whether or not an instruction tostart moving image photography is issued. For example, when the movingimage photography icon 306 is selected or the moving image button isoperated by the user, it is judged that the instruction to start themoving image photography is issued. When it is judged in step S8 thatthe instruction to start the moving image photography is issued, theprocessing proceeds to step S9. When it is judged in step S8 that theinstruction to start the moving image photography is not issued, theprocessing proceeds to step S15.

In step S9, the controller 118 performs the condition-changed movingimage photography. That is, the controller 118 performs imaging by theimaging unit 102 while changing the photography conditions so thatcondition-changed still image data can be generated. Consequently, imagedata (image frames) different in photography conditions are acquired.

For example, in the condition-changed moving image photography set togenerate an HDR still image as the condition-changed still image, thecontroller 118 controls the imaging unit 102 so that image data groupscomprising pieces of image data different in exposure conditions (e.g.exposure time) will be sequentially generated as shown in FIG. 7A. Inthe example of FIG. 7A, the controller 118 controls the imaging unit 102so that overexposure imaging and underexposure imaging will be repeated.An exposure difference in each imaging and others are set in, forexample, in the aforementioned effect check.

In the condition-changed moving image photography set to generate asuper-resolution still image as the condition-changed still image, thecontroller 118 controls the imaging unit 102 so that an image data groupcomprising pieces of image data obtained by controlling the drive unit102 c every timing of imaging to shift the imaging element 102 b in aslight amount (e.g. an amount of 0.5 pixels) as shown in FIG. 7B will besequentially generated. In the example of FIG. 7B, the controller 118controls the drive unit 102 c to shift the imaging element 102 bdownwards to the right, upwards to the right, downwards to the left, andupwards to the left in order.

In step S10, the first image processor 106 a corrects the image data togenerate moving image frames. As shown in FIG. 7A and FIG. 7B, in thecondition-changed moving image photography, the photography condition ofeach moving image frame is different, so that continuity as movingimages is poor. Therefore, in step S10, processing to bring closer theimage qualities of the respective moving image frames to maintaincontinuity as moving images is performed. For example, when it is set togenerate an HDR still image as the condition-changed still image, thefirst image processor 106 a makes a gain adjustment to decrease thebrightness of overexposure image data and increase the brightness ofunderexposure image data, as shown in FIG. 7A. When it is set togenerate a super-resolution still image as the condition-changed stillimage, the first image processor 106 a performs processing to blur theperipheral part of the image data located in a direction in which theimaging element 102 b is shifted at the time of imaging, as shown inFIG. 7B. That is, there is a possibility that the peripheral part of theimage data obtained by shifting the imaging element 102 b in onedirection may not exist in the image data obtained by shifting theimaging element 102 b in the other direction. Therefore, such aperipheral part is blurred to reduce strangeness on moving images.Moreover, when it is set to generate a depth-composed still image as thecondition-changed still image, the first image processor 106 a performscontrast enhancement processing (which may otherwise be edge enhancementprocessing) varying in magnitude according to focus lens position. Thatis, because the state of the contrast of the subject changes due to thedifference of the position of the focus lens, the state of the contrastof the subject is brought closer to a uniform state by the contrastenhancement processing.

Otherwise, the whole screen of each frame is not shown, but a limitedarea of a relatively wide image may be only cut out and displayed. Forexample, when it is set to generate a super-resolution still image asthe condition-changed still image, cut-out processing is performed tocut out a limited area of the image data obtained in the imaging unit102 by the shift direction and shift amount of the imaging element 102 bso that the position of the photography target may not change betweenmoving image frames, whereby even if the image is out of position dueto, for example, the shift of the imaging element 102 b, a correcteddisplay is possible. As above, smoothness of motion is required inmoving images, and even if the frames constituting the condition-changedstill image are obtained with different characteristics and control,smoothness processing and continuity keeping processing to reduceeffects that can be visually recognized in the different processing areperformed without difficulty. In this instance, if sound is collectedand recorded for the photography target, there is a possibility thatacoustic noise at the time of the motion of various mechanisms may beincluded. Therefore, sound is not recorded as it is either, andcondition-changed recording that changes sound collectioncharacteristics (e.g. directivity, gain) and electrically removes noisemay be performed.

In step S11, the controller 118 judges whether or not an instruction toend the moving image photography is issued. For example, when the movingimage photography icon 306 is again selected or the moving image buttonis again operated by the user, it is judged that the instruction to endthe moving image photography is issued. When it is judged in step S11that the instruction to end the moving image photography is not issued,the processing proceeds to step S12. When it is judged in step S11 thatthe instruction to end the moving image photography is issued, theprocessing proceeds to step S14.

In step S12, the controller 118 judges whether or not to perform stillimage recording. Whether or not to perform the still image recording isjudged by the setting performed in step S6. There are various possiblerecording timings of the still image during the moving image photographyto be set by the user in step S6; for example, only the timing in whichan instruction to record the still image is issued by the user, andalways (each timing in which still image data can be generated). When itis judged in step S12 that the still image recording will be performed,the processing proceeds to step S13. When it is judged in step S12 thatthe HDR still image recording will not be performed, the processingreturns to step S9.

In step S13, the controller 118 instructs the image processor 106 togenerate a still image file. In response to this instruction, the secondimage processor 106 b of the image processor 106 composes the image datastored in the storage unit 104 in accordance with the condition-changedmoving image photography to generate condition-changed still image data(e.g. HDR still image data, super-resolution still image data,depth-composed still image data, and panning effect still image data).Then the record processor 106 c generates a still image file from thegenerated condition-changed still image data, and records the generatedstill image file in the recording unit 112. Then the processing returnsto step S9. In the example of FIG. 7A and FIG. 7B, one condition-changedstill image data is generated by the composition of the image dataobtained by two imagings. The composition number and composition ratioof the image data are set, for example, at the time of the effect check.

In step S14, the controller 118 instructs the image processor 106 togenerate a moving image file. In response to this instruction, therecord processor 106 c of the image processor 106 generates a movingimage file from each of the moving image frames stored in the storageunit 104 during the moving image photography, and records the generatedmoving image file in the recording unit 112. Then the processing returnsto step S3.

FIG. 8A is a diagram showing the structure of the moving image fileobtained by the condition-changed moving image photography. As shown inFIG. 8A, a moving image file 601 has moving image data 602 and taginformation 603.

The moving image data 602 is data comprising compressed moving imageframes 602 a. As described above, each of the moving image frames 602 ahas been subjected to processing to maintain continuity as movingimages. Details of this processing may be left as information.

The tag information 603 has file information such as a file name of themoving image file 601. The tag information 603 also has associatedinformation 603 a including information (e.g. a file name) to identifythe condition-changed still image file (e.g. an HDR still image file) tobe associated with each of the moving image frames. Having suchinformation makes it possible to search for the correspondingcondition-changed still image at the time of moving image viewing andplayback, and use the recorded moving image frames to further composestill images.

FIG. 8B is a diagram showing the structure of the condition-changedstill image file (an HDR still image file in the example) obtained bythe condition-changed moving image photography. As shown in FIG. 8B, acondition-changed still image file 604 has condition-changed still imagedata 605 and tag information 606.

The condition-changed still image data 605 is compressedcondition-changed HDR still image data (HDR still image data in theexample). For use in a television, it is not possible to enjoy withoutsound, so that, for example, sound data 605 a obtained at the time ofmoving image recording may be contained in the file. As a result, evenwhen this file is played back on a television or the like, it ispossible to enjoy image viewing together with sound.

The tag information 606 has file information such as a file name of thecondition-changed still image file 604. The tag information 606 also hasassociated information 606 a including information (e.g. a file name) toidentify the moving image file to be associated.

Here, in FIG. 8B, the image data used for the generation of thecondition-changed still image data may be recorded in thecondition-changed still image file 604 in the RAW format. Further, suchRAW data may be recorded in a RAW file different from thecondition-changed still image file 604. In this case, the RAW file alsoneeds to have information to identify the moving image file and thestill image file to be associated with each other. This still image filemay have such a degree of freedom as to allow the specifications to bechanged in accordance with the monitor to which the file is output, andinformation in a data format for use in printers as well as monitors maybe recorded in the same file together.

In the example shown in FIG. 8A and FIG. 8B, the moving image data andthe still image data are recorded in separate files. In contrast, asshown in FIG. 9, the moving image data and the still image data may berecorded in one file. An HDR moving image file 701 in FIG. 9 has movingimage data 702, condition-changed still image data (HDR still image datain the example) 703, and tag information 704. Positional information(e.g. a start address of each data, and a data amount from a startaddress of the file to the start address of each data) of each data toaccess each data in the file is recorded in associated information 704 aof the tag information 704. Although images are only described here, itgoes without saying that sound data collected at the time of imagerecording are also recorded in an associated form in data fortelevisions (both moving images and still images) so that recorded soundcan also be enjoyed at the time of image playback on television.

Here, the explanation returns to FIG. 2A. In step S15 in the case whereit is judged in step S8 that the moving image photography will not beperformed, the controller 118 judges whether or not an instruction tostart still image photography is issued. For example, when the stillimage photography icon 307 is selected or the release button is operatedby the user, it is judged that the instruction to start the still imagephotography is issued. When it is judged in step S15 that theinstruction to start the still image photography is issued, theprocessing proceeds to step S16. When it is judged in step S15 that theinstruction to start the still image photography is not issued, theprocessing returns to step S3.

In step S16, the controller 118 performs condition-changed still imagephotography. That is, the controller 118 performs imaging by the imagingunit 102 the number of times of composition while changing photographyconditions.

In step S17, the controller 118 instructs the image processor 106 togenerate a still image file. In response to this instruction, the secondimage processor 106 b of the image processor 106 composes the image datastored in the storage unit 104 in accordance with the condition-changedmoving image photography to generate condition-changed still image data(e.g. HDR still image data, super-resolution still image data,depth-composed still image data, and panning effect still image data).Then the record processor 106 c generates a still image file from thegenerated condition-changed still image data, and records the generatedstill image file in the recording unit 112. Then the processing returnsto step S3.

In step S18, the controller 118 judges whether or not an instruction tostart moving image photography is issued. When it is judged in step S18that the instruction to start the moving image photography is issued,the processing proceeds to step S19. When it is judged in step S18 thatthe instruction to start the moving image photography is not issued, theprocessing proceeds to step S22.

In step S19, the controller 118 performs normal moving imagephotography. That is, the controller 118 performs imaging by the imagingunit 102 without changing photography conditions. In this instance, thecontroller 118 also collects sound with the microphone 120 and recordsthe sound so that the sound during the image recording can also beenjoyed.

In step S20, the controller 118 judges whether or not an instruction toend the moving image photography is issued. It is judged that theinstruction to end the moving image photography is issued. When it isjudged in step S20 that the instruction to end the moving imagephotography is not issued, the processing returns to step S19. When itis judged in step S20 that the instruction to end the moving imagephotography is issued, the processing proceeds to step S21.

In step S21, the controller 118 instructs the image processor 106 togenerate a normal moving image file. In response to this instruction,the record processor 106 c of the image processor 106 generates a normalmoving image file from the moving image frames stored in the storageunit 104 during the normal moving image photography, and records thegenerated normal moving image file in the recording unit 112. Then theprocessing returns to step S3.

In step S22 in the case where it is judged in step S18 that theinstruction to start the moving image photography is not issued, thecontroller 118 judges whether or not an instruction to start still imagephotography is issued. For example, when the still image photographyicon 307 is selected or the release button is operated by the user, itis judged that the instruction to start the still image photography isissued. When it is judged in step S22 that the instruction to start thestill image photography is issued, the processing proceeds to step S23.When it is judged in step S22 that the instruction to start the stillimage photography is not issued, the processing proceeds to step S25.

In step S23, the controller 118 performs normal still image photography.That is, the controller 118 performs imaging once by the imaging unit102 in accordance with the exposure conditions set in step S6. Then theprocessing proceeds to step S24.

In step S24, the controller 118 instructs the image processor 106 togenerate still image data. In response to this instruction, the secondimage processor 106 b of the image processor 106 generates normal stillimage data from the image data stored in the storage unit 104 by thenormal still image photography. After the generation of the normal stillimage data, the record processor 106 c records the generated normalstill image file in the recording unit 112. Then the processing returnsto step S3.

In step S25, the controller 118 judges whether or not a returninstruction is issued by the user. For example, when the return icon 308is selected by the user, it is judged that the return instruction isissued. When it is judged in step S25 that the return instruction is notissued by the user, the processing returns to step S3. When it is judgedin step S25 that the return instruction is issued by the user, theprocessing returns to step S1.

In step S26 in the case where it is judged in step S2 that the operationmode is not the photography mode, the controller 118 performs processingother than that of the photography mode. After the other processing, theprocessing returns to step S1.

FIG. 10A and FIG. 10B are flowcharts showing the other processing. Instep S201, the controller 118 judges whether or not the currentoperation mode is the playback mode. When it is judged in step S201 thatthe operation mode of the imaging apparatus 100 is not the playbackmode, the processing proceeds to step S202. When it is judged in stepS201 that the operation mode of the imaging apparatus 100 is theplayback mode, the processing proceeds to step S204.

In step S202, the controller 118 judges whether or not the currentoperation mode is the communication mode. When it is judged in step S202that the operation mode of the imaging apparatus 100 is thecommunication mode, the processing proceeds to step S203. When it isjudged in step S202 that the operation mode of the imaging apparatus 100is not the communication mode, the processing in FIG. 10A and FIG. 10Bends. In this case, the processing returns to step S1 in FIG. 2A.

In step S203, the controller 118 performs the processing of thecommunication mode. In the processing of the communication mode, thecontroller 118 controls the communication unit 114 to perform suchprocessing as to send the image file recorded in the recording unit 112to an external device or receive the image file recorded in the externaldevice. After the processing of the communication mode, the processingin FIG. 10A and FIG. 10B ends. In this case, the processing returns tostep S1 in FIG. 2A.

In step S204, the controller 118 controls the playback unit 108 todisplay a list of the image files recorded in the recording unit 112. Inresponse to this instruction, the playback unit 108 displays, on thedisplay 110, for example, thumbnail images showing the list of the imagefiles recorded in the recording unit 112. Then the processing proceedsto step S205.

In step S205, the controller 118 judges whether or not a moving imagefile is selected from the displayed list by the user. When it is judgedin step S205 that a moving image file is selected by the user, theprocessing proceeds to step S206. When it is judged in step S205 that amoving image file is not selected by the user, the processing proceedsto step S216.

In step S206, the controller 118 judges whether or not the imagingapparatus 100 is connected to an external display device (e.g. thetelevision 200). When it is judged in step S206 that the imagingapparatus 100 is connected to the external display device, theprocessing proceeds to step S207. When it is judged in step S206 thatthe imaging apparatus 100 is not connected to the external displaydevice, the processing proceeds to step S213.

In step S207, the controller 118 instructs the playback unit 108 to sendthe moving image frames corresponding to the selected moving image fileto the external display device. Accordingly, the first playbackprocessor 108 a of the playback unit 108 sends the moving image framesto the television 200 which is the external display device via thecommunication unit 114. Consequently, in the television 200 which is theexternal display device, for example, moving images are played back asshown in FIG. 11. In this instance, the images are output while beingplayed back, so that the playback processors have a function to suitablyplay back the file which has been recorded with special elaboration asshown in FIG. 8A, FIG. 8B, FIG. 9, and others. For example, sound andimages are sent to the television if the television is connected by asystem such as HDMI.

In step S208, the controller 118 judges whether or not it is the timingof playing back a still image. For example, when there is still imagedata (condition-changed still image data or normal still image data)associated with the moving image frames being played back, it is judgedthat it is the timing of playing back the still image. When it is judgedin step S208 that it is the timing of playing back the still image, theprocessing proceeds to step S209. When it is judged in step S208 that itis not the timing of playing back the still image, the processingproceeds to step S212.

In step S209, the controller 118 judges whether or not thecondition-changed still image data is associated with the moving imageframes being played back. When it is judged in step S209 that thecondition-changed still image data is associated with the moving imageframes being played back, the processing proceeds to step S210. When itis judged in step S209 that the condition-changed still image data isnot associated with the moving image frames being played back, theprocessing proceeds to step S211.

In step S210, the controller 118 instructs the playback unit 108 to sendthe condition-changed still image data associated with the moving imageframes being played back to the external display device. Accordingly,the second playback processor 108 b of the playback unit 108 sends thecondition-changed still image data to the television 200 which is theexternal display device via the communication unit 114. Consequently, inthe television 200 which is the external display device, acondition-changed still image is played back. The condition-changedstill image may be played back after the end of the playback of themoving image frames, or may be reduced in size and then played back overthe moving image frames being played back.

In step S211, the controller 118 instructs the playback unit 108 to sendthe normal still image data associated with the moving image framesbeing played back to the external display device. Accordingly, thesecond playback processor 108 b of the playback unit 108 sends thenormal still image data to the television 200 which is the externaldisplay device via the communication unit 114. Consequently, in thetelevision 200 which is the external display device, a normal stillimage is played back. The normal still image may be played back afterthe end of the playback of the moving image frames, or may be reduced insize and then played back over the moving image frames being playedback.

In step S212, the controller 118 judges whether or not an instruction toend the playback of the moving images is issued. For example, when areturn icon 801 shown in FIG. 11 is selected, it is judged that theinstruction to end the playback of the moving images is issued. When itis judged in step S212 that the instruction to end the playback of themoving images is not issued, the processing returns to step S206. Whenit is judged in step S212 that the instruction to end the playback ofthe moving images is issued, the processing returns to step S204.

In step S213 in the case where it is judged in step S206 that theimaging apparatus 100 is not connected to the external display device,the controller 118 instructs the playback unit 108 to display the movingimage frames corresponding to the selected moving image file on thedisplay 110. Accordingly, the first playback processor 108 a of theplayback unit 108 displays the moving image frames on the display 110.

In step S214, the controller 118 judges whether or not it is the timingof playing back a still image. For example, when there is still imagedata associated with the moving image frames being played back, it isjudged that it is the timing of playing back the still image. When it isjudged in step S214 that it is the timing of playing back the stillimage, the processing proceeds to step S215. When it is judged in stepS214 that it is not the timing of playing back the still image, theprocessing proceeds to step S212.

In step S215, the controller 118 instructs the playback unit 108 todisplay, on the display 110, a still image associated with the movingimage frames being played back. Accordingly, the second playbackprocessor 108 b of the playback unit 108 displays the still image on thedisplay 110. Consequently, the still image is played back on the display110.

In step S216 in the case where it is judged in step S205 that a movingimage file is not selected by the user, the controller 118 judgeswhether or not a still image file is selected from the displayed list bythe user. When it is judged in step S216 that a still image file isselected by the user, the processing proceeds to step S217. When it isjudged in step S216 that a still image file is not selected by the user,the processing proceeds to step S223.

In step S217, the controller 118 judges whether or not the imagingapparatus 100 is connected to an external display device (e.g. thetelevision 200). When it is judged in step S217 that the imagingapparatus 100 is connected to the external display device, theprocessing proceeds to step S218. When it is judged in step S217 thatthe imaging apparatus 100 is not connected to the external displaydevice, the processing proceeds to step S219.

In step S218, the controller 118 instructs the playback unit 108 tosend, to the external display device, the still image data(condition-changed still image data or normal still image data) includedin the selected still image file. Accordingly, the second playbackprocessor 108 b of the playback unit 108 sends the still image data tothe television 200 which is the external display device via thecommunication unit 114. Consequently, in the television 200 which is theexternal display device, a still image is played back.

In step S219, the controller 118 instructs the playback unit 108 to playback, on the display 110, the still image data (condition-changed stillimage data or normal still image data) included in the selected stillimage file. Accordingly, the second playback processor 108 b of theplayback unit 108 displays the still image on the display 110.

In step S220, the controller 118 judges whether or not to make an effectcheck. For example, when the HDR effect check icon displayed on thedisplay 110 during the playback of the still image is selected, it isjudged that the effect check will be made. When it is judged in stepS220 that the effect check will be made, the processing proceeds to stepS221. When it is judged in step S220 that the effect check will not bemade, the processing proceeds to step S222.

In step S221, the controller 118 performs the processing of the effectcheck. The processing of the effect check is substantially the same asthe processing in FIG. 4 described above. In the effect check in thephotography mode, image data for the effect check is acquired byimaging. In contrast, in the effect check in the playback mode, imagedata for the effect check can be acquired in any manner. After the endof the effect check, the processing proceeds to step S222.

In step S222, the controller 118 judges whether or not an instruction toend the playback of the still image is issued. When it is judged in stepS222 that the instruction to end the playback of the still image is notissued, the processing returns to step S217. When it is judged in stepS222 that the instruction to end the playback of the still image isissued, the processing returns to step S204.

In step S223, the controller 118 judges whether or not to end theprocessing in the playback mode. For example, when a return instructionis issued by the user during the display of the list, it is judged thatthe processing in the playback mode will be ended. When it is judged instep S223 that the processing in the playback mode will not be ended,the processing returns to step S204. When it is judged in step S223 thatthe processing in the playback mode will not be ended, the processing inFIG. 10A and FIG. 10B ends. In this case, the processing returns to stepS1 in FIG. 2A.

As described above, in the present embodiment, it is possible togenerate a condition-changed still image which is generated by thecomposition of image data different in photography conditions, byperforming imaging while changing the photography conditions duringmoving image photography. Consequently, moving images can be moreattractive contents.

Furthermore, in the present embodiment, moving image frames aregenerated after image data obtained by multiple imaging different inphotography conditions is subjected to processing that maintainscontinuity of moving images. This makes it possible to perform bothdisplay suited to the specifications of the display 110 of the imagingapparatus and display suited to the specifications of the television200.

While the present invention has been described above on the basis of theembodiment, the present invention is not limited to the embodimentdescribed above, and it should be understood that various modificationsand applications can be made within the scope of the spirit of thepresent invention. In the technique according to the present embodiment,the condition-changed moving image photography can be replaced withthrough-image display. In this case, various still images such as an HDRstill image, a super-resolution still image, and a depth-composed stillimage can be displayed by the timing specified by the user during thethrough-image display. Such processing is applicable to imagingapparatuses such as an endoscope device, a microscope device, and amonitoring device.

Each process according to the embodiment described above can also bestored as a program executable by the controller 118 which is acomputer. Otherwise, each process according to the embodiment describedabove can be stored and distributed in a recording medium of an externalstorage device such as a magnetic disk, an optical disk, or asemiconductor memory. The controller 118 then reads the program storedin the recording medium of the external storage device, and theoperation of the controller 118 is controlled by the read program,whereby the controller 118 can execute the processes described above. Asapplicability, whether or not the result of display is satisfactory canbe input by manual operation, biological determination, or the like sothat the user's taste will be learned accordingly. That is, control thatcooperates with artificial intelligence may be used. In this case,similar advantageous effects can be obtained even without a step-by-stepprogram.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An imaging apparatus comprising: an imaging unitwhich sequentially acquires image frames whose photography conditionshave been changed; a first image processor which generates moving imagedata by performing image processing to maintain continuity between theimage frames obtained by changing the photography conditions, duringrecording of a moving image; a second image processor which composes theimage frames whose photography conditions have been changed, to generatecondition-changed still image data; and a record processor configured torecord the moving image data and the condition-changed still image datain a single file so that the moving image data and the condition-changedstill image data are associated with each other.
 2. An imaging apparatuscomprising: an imaging unit which sequentially acquires image datagroups comprising pieces of image data different in photographyconditions; a first image processor which generates moving image datacomprising moving image frames formed by each of the pieces of the imagedata; a second image processor which composes pieces of image databelonging to each of the image data groups or pieces of image data thatdo not belong to the image data groups, to generate condition-changedstill image data; and a record processor configured to record the movingimage data and the condition-changed still image data in a single fileso that the moving image data and the condition-changed still image dataare associated with each other, wherein the first image processorgenerates moving image frames after subjecting each of the pieces of theimage data to image processing which maintains continuity between theimage frames during recording of a moving image.
 3. The imagingapparatus according to claim 2, wherein the photography conditions arean exposure difference between the pieces of the image data, and theimage processing which maintains continuity is a gain adjustment tobring exposure amounts of the pieces of the image data to a uniformexposure amount.
 4. The imaging apparatus according to claim 2, whereinthe photography conditions are a shift direction of an imaging elementincluded in the imaging unit, and the image processing which maintainscontinuity is blurring processing for a peripheral portion of the pieceof the image data positioned in the shift direction of the imagingelement, or cut-out processing to cut out the piece of the image data inaccordance with the shift direction of the imaging element so that theposition of a target of photography does not change.
 5. An imagingmethod comprising: sequentially acquiring, by an imaging unit, imageframes whose photography conditions have been changed; generating movingimage data by performing image processing to maintain continuity betweenthe image frames obtained, during recording of a moving image, bychanging the photography conditions; composing the image frames whosephotography conditions have been changed, to generate condition-changedstill image data; and recording the moving image data and thecondition-changed still image data in a single file so that the movingimage data and the condition-changed still image data are associatedwith each other.
 6. An imaging method comprising: sequentiallyacquiring, by an imaging unit, image data groups comprising pieces ofimage data different in photography conditions; generating moving imagedata comprising moving image frames formed by each of the pieces of theimage data after subjecting each of the pieces of the image data toimage processing which maintains continuity between the image framesobtained during recording of a moving image; composing image databelonging to each of the image data groups or image data that do notbelong to the image data groups, to generate condition-changed stillimage data; and recording the moving image data and thecondition-changed still image data in a single file so that the movingimage data and the condition-changed still image data are associatedwith each other, wherein the first image processor generates movingimage frames after subjecting each of the pieces of the image data toimage processing which maintains continuity between the image framesduring recording of a moving image.
 7. The imaging apparatus of claim 1wherein the photography conditions of the image frames sequentiallyacquired are changed cyclically, and wherein the image processingperformed to maintain continuity between the image frames obtainedremoves or reduces the effect of the cyclically changed photographyconditions thereby generating a moving image file with improvedcontinuity.
 8. The imaging method of claim 5 wherein the photographyconditions of the image frames sequentially acquired are changedcyclically, and wherein the image processing performed to maintaincontinuity between the image frames obtained removes or reduces theeffect of the cyclically changed photography conditions therebygenerating a moving image file with improved continuity.